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ABSTRACT 

Intelligence  cells  have  to  process  and  evaluate  current  information  to  deduce  timely  and  most  reliable  an 
appropriate  picture  of  the  battlespace.  The  presented  approach  of  knowledge  based  information  fusion  is 
focussing  on  the  heuristic  human  evaluation  process.  Military  information  processing  is  modelled  as  a 
context  dependent  and  template-based  heuristic  reasoning  process  and  the  real  task  is  modelled  by  a 
closed  world  representation.  The  developed  method  of  information  fusion  is  basically  independent  from  a 
specific  military  scenario.  It  is  based  on  the  assumption  that  forces  are  organised  in  a  structured  manner 
and  that  they  operate  in  a  military  reasonable  and  typical  way.  Respective  rules  and  doctrines  can  be  used 
as  matching  templates  for  knowledge  based  reasoning.  By  this  approach,  the  analysis  and  fusion  of 
incomplete  and  imperfect  information  of  military  reports  and  background  knowledge  can  be  supported 
substantially  in  an  automated  system. 

Keywords:  Template  based  reasoning,  heuristic  reasoning,  situation  awareness,  military  intelligence, 
decision  support 


1.0  INTRODUCTION 

1.1  The  Objective  of  Military  Intelligence 

In  military  command  and  control  a  most  accurate  situational  awareness  of  the  battlespace  is  essential  prior 
to  all  decisions  and  activities.  This  is  a  basic  military  requirement  which  is  independent  from  the  ever 
changing  and  variety  of  potential  conflicts.  Within  the  global  and  military  information  environment, 
the  challenge  to  future  military  operations  will  be  to  manage  large  volumes  of  information  rapidly  to 
portray  the  results  in  a  prompt  and  meaningful  manner.  Intelligence  cells  have  to  process  and  evaluate 
incoming  reports  to  deduce  timely  and  most  reliable  an  appropriate  picture  of  the  battlespace.  This  task 
includes  to  determine  the  most  actual  location,  strength,  and  activities  of  all  engaged  faction  forces  and  to 
deduce  their  likely  intensions.  A  wide  variety  of  information  produced  by  the  full  spectrum  of  sensors  and 
human  sources  has  to  be  collected,  filtered,  processed  and  disseminated. 

1.2  The  Military  Intelligence  Cycle 

This  is  done  in  a  structured  and  systematic  series  of  operations  which  is  called  the  Intelligence  Cycle  (IC). 
It  is  the  framework  within  which  four  discrete  operations  are  conducted  culminating  in  the  distribution  of 
the  finished  intelligence  product.  The  sequence  is  cyclic  in  nature  since  intelligence  requires  constant 
reappraisal  and  updating  if  it  is  to  remain  current  and  relevant  to  a  commander’s  needs.  The  operations  are 
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discrete  because,  as  information  begins  to  flow,  is  processed  and  disseminated  as  intelligence, 
the  operations  will  overlap  and  coincide  so  that  they  are  being  conducted  concurrently  and  continuously 
rather  than  sequentially.  The  operations  or  ’phases’  are  shown  in  Figure  1  and  are  defined  by  the  NATO 
Glossary  of  Terms  and  Definitions  (AAP-6)  [1 1]  as  follows: 

1)  Direction:  “Determination  of  intelligence  requirements,  planning  the  collection  effort,  issuance  of 
orders  and  requests  to  collection  agencies  and  maintenance  of  a  continuous  check  on  the 
productivity  of  such  agencies.” 

2)  Collection:  “The  exploitation  of  sources  by  collection  agencies  and  the  delivery  of  the  information 
obtained  to  the  appropriate  processing  unit  for  use  in  the  production  of  intelligence.” 

3)  Processing :  “The  production  of  intelligence  through  collation,  evaluation,  analysis,  integration 
and  interpretation  of  information  and/or  other  intelligence.” 

4)  Dissemination :  “The  timely  conveyance  of  intelligence,  in  an  appropriate  form  and  by  any 
suitable  means,  to  those  who  need  it”. 


Figure  1:  The  Intelligence  Cycle. 


This  paper  will  present  an  approach  to  give  automated  support  to  the  analysis  and  integration  steps  of  the 
“Processing”  phase  which  in  military  terms  is  the  phase  where  information  is  converted  to  intelligence. 

1.3  The  Objectives  of  Information  Fusion  in  Intelligence  Processing 

Current  information  fed  into  intelligence  cells  by  reports  originating  from  all  different  sources  of 
reconnaissance  (HUMINT,  SIGINT,  IMINT,  OSINT)  are  related  mostly  to  particular  observable  elements 
of  the  mission's  area  of  interest.  Each  of  these  reports  gives  a  narrow  view  on  a  specific  aspect  of  the 
respective  local  situation,  for  example  a  small  group  of  moving  vehicles,  the  detection  of  an  abandoned 
training  area,  or  the  theft  of  explosives.  The  military  commander  however  is  interested  in  a  comprehensive 
and  complete  picture  of  the  overall  situation,  where  all  available  information  is  taken  into  account  and  all 
relevant  facts  are  fused  to  an  integrated  description  of  the  situation.  Such  a  deduced  picture  is  supposed  to 
reveal  objects  which  are  more  complex  and  represent  e.g.  higher  military  hierarchy’s  units  and/or 
activities  on  an  appropriate  operational  level,  e.g.  march  columns,  indicating  the  mobilisation  of  higher 
formations  and  (massive)  force  deployment.  Such  high  level  domain  objects  themselves  are  not  observable 
by  the  means  of  reconnaissance.  They  can  only  be  deduced  from  the  current  information  of  the  developing 
situation  and  a  sufficient  knowledge  about  the  information  context  of  the  domain.  Thus  the  sequence  of 
processing  steps  of  the  "Processing"  phase  of  the  IC  produces  an  increasingly  aggregated  and  fused  picture 
of  the  situation  with  a  growing  level  of  abstraction  of  the  constituent  objects.  Starting  with  physical 
objects  like  tanks  or  a  stolen  vehicle  they  end  with  abstract  objects  like  the  situation  of  the  battlespace  or  a 
threat  assessment  of  a  bomb  attack.  These  processing  steps  are  related  to  level  2  (situation  assessment)  and 
level  3  (impact  assessment)  fusion  problems,  according  to  the  definitions  of  the  revised  Data  Fusion 
Model,  which  is  maintained  by  the  Data  Fusion  Group  of  the  Joint  Directors  of  Laboratories  (JDL) 
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Technical  Panel  for  C3I  [16].  In  level  2  an  observed  object  is  associated  to  a  particular  unit  of  the 
adversary  forces  (see  section  3.2  Classification),  hypotheses  of  the  role  of  this  unit  in  the  concert  of  all 
ongoing  activities  are  established  and  a  most  accurate  description  of  the  overall  situation  is  developed 
(see  section  3.3  Template  Based  Aggregation).  This  is  done  predominantly  during  the  analysis  and 
integration  steps  of  the  ’’Processing”  phase  of  the  IC  which  are  the  main  focus  in  our  own  project  [3]  and 
of  this  paper.  In  level  3  any  impact  of  the  determined  situation  on  own  interests,  especially  the  potential 
threat  on  own  and  friendly  forces  is  assessed.  This  is  done  within  the  interpretation  step  of  the 
’’Processing”  phase  and  not  covered  by  this  paper.  Some  aspects  of  threat  assessment  are  considered  in 
[10]. 

A  comprehensive  introduction  to  automation  of  data  fusion,  especially  to  tactical  data  fusion,  can  be  found 
in  [1].  In  [5]  a  general  presentation  of  different  aspects  of  information  fusion  in  different  domains  is  given. 


2.0  FUNDAMENTALS  OF  THE  HEURISTIC  INTELLIGENCE  PROCESSING 

A  main  intellectual  and  cognitive  part  in  the  production  of  military  intelligence  is  done  by  comparing  the 
partial  information  aspects  given  by  the  reported  low  level  objects  with  the  so  far  deduced  perceived 
picture  of  the  current  situation,  relating  them  to  the  most  likely  aspects,  aggregate  them  to  high  level 
situation  objects,  and  finally  integrate  these  into  the  picture  of  the  situation.  As  a  metaphor  this 
information  fusion  problem  is  compared  by  Antony  [1]  to  the  jigsaw  puzzle  problem.  The  very  difference 
to  the  game  situation  however  is,  that,  as  nobody  knows  the  real  battlespace  situation,  the  intelligence  cell 
has  to  deduce  the  situation  picture  from  the  pieces  without  knowing  a  picture  of  the  real  solution  as  it  is 
available  in  a  game.  To  overcome  this  lack  of  a  guiding  true  solution  the  processing  steps  are  based  on 
knowledge  of  the  background  of  the  situation  which  is  called  basic  intelligence.  The  AAP-6  [11] 
definition  of  basic  intelligence  is:  “Intelligence,  on  any  subject,  which  may  be  used  as  reference  material 
for  planning  and  as  a  basis  for  processing  subsequent  information  or  intelligence”.  All  information  about 
engaged  factions  describing  their  structure  and  strength,  their  equipment  and  their  initial  deployment  is  of 
interest.  It  is  continually  updated  in  peace  and  in  the  course  of  operations.  The  principal  use  of  basic 
intelligence  is  to  set  the  scene  at  the  outset  of  operations  and  to  meet  intelligence  requirements  dealing 
with  unchanging  facts  such  as  battlespace  terrain  and  climate  which  may  be  raised  in  answer  to  new 
requirements  in  the  course  of  an  operation  [12]. 

2.1  Basic  Assumptions 

Up  to  now  the  production  of  military  intelligence  is  based  on  the  experience  that  all  hostile  forces,  groups 
or  elements  are  organised  and  have  an  organisational  structure,  operate  in  mission  co-ordinated  and,  with 
respect  to  their  objectives,  in  a  determined  and  reasonable  way.  Military  leaders  understand  war  as  a 
highly  complex  system  of  separable  activities,  as  it  is  shown  in  Figure  2  [13]. 
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WAR 


Figure  2:  War  as  a  Structured  System  of  Separable  Activities. 


Individual  missions  and  single  activities  will  be  planned  based  on  some  kind  of  doctrines,  modus  operandi 
and  rules  reflecting  their  structure  and  their  organisation,  so  they  can  be  distinguished  from  each  others 
and  recognised  by  significant  information.  This  assumption  has  been  confirmed  by  experience  not  only  for 
conflicts  with  regular  military  forces  but  also  is  applicable  for  para-military  factions,  certain  criminal 
activities  [1],  [17].  Even  in  Counter  Terrorist  Operations  the  search  for  anomalies  in  patterns  of  standard 
behaviour  as  significant  facts  for  an  identification  of  terrorists  can  be  interpreted  as  a  template-based 
approach  [14]. 

Doctrines  and  rules  proved  to  be  a  good  basis  for  military  success  but  they  also  give  a  good  basis  for  an 
approach  for  information  processing  and  situation  awareness.  Intelligence  officers  practise  by  default  a 
method  of  heuristic  reasoning  which  relies  on  their  knowledge  about  standards  and  rules  of  the  adversary 
and  the  assumption  that  the  opposing  forces  will  act  according  to  these  regulations  and  constraints.  It  is 
common  military  experience  and  expert  knowledge,  that  the  production  of  intelligence  can  be  based  on 
putting  current  information  into  such  an  information  context.  Information  dominance  is  a  battle  winning 
factor,  thus  it  is  a  main  concern  of  every  intelligence  activity  to  build  up  sufficient  knowledge  about  the 
adversary.  The  lack  of  basic  intelligence  or  background  knowledge  does  not  prove  the  absence  of  e.g.  an 
organisational  structure  or  principles  of  behaviour,  it  only  proves  the  urgent  need  of  reconnaissance. 
A  huge  variety  of  information  of  every  kind  is  needed  and  a  typical  list  of  basic  intelligence  categories 
will  include: 

•  organisational  structure  and  equipment, 

•  operational  doctrines  and  activity  patterns, 

•  political  parties  and  political  structures  (coalitions), 

•  sociologic,  religious,  and  ethnic  information, 
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•  local  places  and  habits, 

•  geography,  infrastructure,  climate,  ...  . 

These  information  aspects  span  the  information  context  space  in  which  information  processing  is  done  in 
intelligence  cells.  The  relevance  of  the  individual  aspects  may  be  dependent  from  the  specific  scenario  and 
situation  and  the  importance  of  a  certain  category  of  information  is  difficult  to  assess  as  long  as  no 
sufficient  understanding  of  the  overall  situation  has  been  established.  Information  concerning  the 
organisation  and  strength  of  adversary  forces  always  will  be  of  fundamental  interest.  The  information 
about  the  historical,  religious  and  social  situation  of  the  population  gained  importance  during  the  last  years 
in  the  course  of  'Operation  Other  Than  War'  (OOTW)  missions,  e.g.  during  the  Balkan  conflict  [17]. 
Depending  on  the  evolving  situation  and  the  accordingly  changing  own  information  and  intelligence 
requirements  the  list  of  necessary  categories  may  be  expanded  and  the  importance  of  its  elements  may 
change.  The  real  information  fusion  problem  in  principal  is  an  open  context  problem,  but  in  a  real  mission 
the  full  information  context  never  will  be  known  completely. 

The  cognitive  processes  of  the  intelligence  officers  about  the  supposed  default  behaviour  of  the  enemy  are 
typically  based  on  a  limited  number  of  intelligence  aspects  thus  defining  a  closed  world  model  of  the 
problem  space  and  they  constitute  an  experience  based  default  reasoning  process.  Default  behaviour  of  an 
adversary  allows  to  develop  generic  doctrinal  templates  of  his  activities  which  have  to  be  adapted  to  real 
mission  environment  and  to  define  significant  information  indicating  a  certain  military  activity  which  is 
related  to  this  behaviour.  A  major  part  of  the  information  processing  in  intelligence  cells  is  done  based  on 
this  heuristic  method  which  is  the  fundamental  idea  of  template  based  information  fusion.  (For  a  basic 
discussion  about  doctrinal  templating  see  also  Antony  [1].)  The  template-based  default  reasoning  approach 
is  the  basis  to  develop  automated  evaluation  algorithms  to  support  the  human  intelligence  processes. 

The  matrix  in  Table  1  shows  the  functional  template  of  a  march  column  from  a  classical  scenario  [3]. 
It  describes  in  general  the  components  of  a  march  column  by  their  military  hierarchy  and  function.  It  is 
used  in  our  experimental  system,  beside  templates  on  the  spatial  structure  and  dynamic  features,  as  a  basis 
for  the  aggregation  of  single  reports  about  moving  troops  into  march  columns. 


Table  1:  Functional  Template  of  a  March  Column 


Head  of  column 

Inner  column  components 

End  of  column 

Type  of  troops 

reconnaissance 

security  units 

combat  units 

combat  support  units 

air  defence  and  security 

logistics 

air  defence  and  security 

Branch 

armoured  reconnaissance 

antitank  defence 

Armour,  mechanised  infantry  ,  non 
armoured  infantry,  multi  rocket  launcher, 
artillery,  engineer 

AA,  SAM 

transportation  & 
maintenance 

AA,  SAM 

Command  level 

<  platoon  or  company 

<  battalion 

<  platoon,  company  or 

battery 

Type  of  vehicle 

APC,  ARV 

MBT,  APC,  AAA 

VEH,  AAA 

(AA  Anti  Air;  AAA  Armoured  Anti  Air;  APC  Armoured  Personnel  Carrier;  ARV  Armoured 
Reconnaissance  Vehicle;  MBT  Main  Battle  Tank;  SAM  Surface  Air  Missile;  VEH  Vehicle) 


More  complex  template  structures  in  the  context  of  low  intensity  conflicts  have  been  used  in  our  recent 
analysis  on  information  fusion  for  threat  assessment  [9]  [10].  The  underlying  information  environment 
was  expanded  significantly  and  structured  into  ability  and  threat  related  aspects. 
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2.2  Quality  of  the  Current  Information 

Topical  information  about  the  situation  given  by  reports  may  be  imperfect  in  various  respects  and  it  is 
typically  incomplete,  imprecise,  uncertain,  and  vague  and  the  sequence  of  incoming  reports  will  not 
necessarily  be  in  a  chronological  order.  (For  a  short  definition  of  the  different  aspects  of  defectiveness  see 
[5])  It  is  a  main  benefit  of  the  template-based  approach  that  some  deficiencies  from  incompleteness  and 
imprecision  of  the  current  information  can  be  compensated  partly.  This  is  caused  by  the  general  and  more 
qualitative  nature  of  the  template -based  information  fusion  focussing  on  the  deduction  of  more  abstract 
high  level  objects  of  the  problem  domain.  A  lot  of  calculations  concerning  the  correlation  of  two  low  level 
objects  cannot  be  carried  out  properly  because  of  the  imperfectness  of  the  information  or  would  only 
produce  less  usable  ambiguous  results.  In  the  context  of  the  template-based  aggregation  approach  they 
could  be  avoided.  To  give  an  example,  it  is  not  necessary  to  show  that  two  low  level  elements  observed  at 
different  times  at  different  places  are  identical  objects.  The  military  interest  is  to  know  whether  or  not  they 
belong  to  the  same  higher  level  element  or  activity,  because  this  will  answer  the  question  about  location, 
strength  and  behaviour  of  the  adversary.  Target  correlation  is  not  a  the  goal  of  level  2  fusion  problems 
e.g.  at  a  brigade  level,  but  object  aggregation  and  role  identification.  Template-based  aggregation  is  a 
suitable  method  to  cope  with  this  problem. 


3.0  KNOWLEDGE-BASED  PROCESSING  METHODS 

The  principle  feasibility  of  automated  information  fusion  for  the  support  of  intelligence  processing  has 
been  proven  in  a  former  project  which  was  based  on  a  classical  cold  war  scenario  [4].  Our  actual  approach 
of  a  knowledge  based  information  fusion  is  focussing  on  the  heuristic  basics  of  the  human  cognitive 
processing  of  reports  in  an  army  intelligence  cell  and  broadening  the  view  to  actual  military  missions. 
Military  information  fusion  is  modelled  as  a  context-dependent  and  template-based  heuristic  reasoning 
process.  The  aspects  of  spatial  and  temporal  reasoning  which  are  included  in  our  ongoing  research  are 
covered  in  [10]. 

The  hereafter  presented  knowledge-based  methods  can  be  interpreted  as  parts  of  the  IC  processing  phase 
as  it  has  been  analysed  and  modelled  by  NATO  RTO  IST-015/  TASK  GROUP  004  ON  INFORMATION 
FUSION  (2000  -  2002)  [15]  and  may  particularly  be  related  to  the  Collation  and  Analysis  &  Integration 
steps,  as  they  are  shown  in  Figure  3. 


Products 


Correlation 

Aggregation 


Classification 


Message  Processing 


Figure  3:  Intelligence  Cycle  Processing  Phase  -  Representation  of  Processes  [15]. 
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3.1  Context  Dependent  Collation 

During  the  collation  step  of  the  processing  phase  of  the  IC,  a  first  semantic  analysis  is  performed 
associating  any  new  report  to  an  appropriate  category  of  intelligence.  The  number  and  definition  of  the 
categories  depends  on  the  type  of  operation  which  is  conducted.  These  categories  are  deduced  from  the 
commanders  intelligence  requirements  and  define  the  main  situation  aspects  and  the  information  context 
which  has  to  be  covered  to  be  able  to  give  a  description  of  the  situation  that  sufficiently  reflects  the 
abilities  and  intentions  of  the  opposite  parts  and  to  make  an  assessment.  The  association  of  a  single  report 
with  a  category  of  information  is  just  a  loose  correlation.  No  further  semantic  interpretation  is  done  at  this 
step.  Formatted  reports  do  name  a  main  category  of  information  by  the  use  of  key  words.  Structured  text 
reports  also  support  this  categorisation.  For  free  text  reports  there  has  to  be  a  pre-processing  to  exploit  the 
semantic  content.  It  is  a  premise  of  our  approach  that  a  semantic  understanding  and  categorisation  of  the 
current  information  is  given  prior  to  our  processing,  as  it  is  e.g.  possible  with  formatted  reports.  If  this 
premise  is  not  true,  a  substantial  automated  support  to  an  object  oriented  processing  and  information 
fusion  is  doubtful. 

3.2  Classification 

Classification  is  one  of  the  fundamental  tasks  of  an  intelligence  cell.  There  is  a  strong  interest  to  get  to 
know  which  unit  of  the  adversary  forces  was  engaged  when  an  activity  or  event  was  observed,  answering 
the  initial  question:  ’’Who  or  what  is  it?"  Thus  a  basic  functionality  in  information  processing  is  the 
correlation  of  observed  situation/domain  elements  to  a  domain  model  from  basic  intelligence.  For  a  more 
detailed  definition  of  classification  see  [15].  A  generic  solution  method  for  this  matching  problem  was 
developed,  which  is  independent  from  the  special  scenario  [7].  The  algorithm  takes  e.g.  the  known 
structure  of  the  opposing  forces  and  their  list  of  equipment  as  initial  parameters.  Thus  the  precondition  of 
the  method  is  the  same  as  for  the  real  processing.  Without  any  knowledge  about  the  opposite  organisation 
no  classification  is  possible.  It  is  proven  that  the  computation  based  on  this  method,  which  is  a  logic  two- 
rule  system,  is 

•  complete,  so  every  solution  to  a  matching  problem  is  calculated, 

•  correct,  so  every  calculated  solution  solves  the  matching  problem, 

•  finite,  that  is  the  computation  stops  in  finite  time. 

The  combinatorial  complexity  problems  of  the  pure  theoretic  algorithm  may  be  reduced  by  the  use  of 
constraints,  which  can  be  dependent  or  independent  from  the  domain.  They  are  expressed  as  constraining 
rules  and  enhance  the  performance  of  an  algorithm  based  on  the  abstract  method  and  the  quality  of  the 
results.  A  scenario  independent  rule  is,  to  start  by  matching  first  the  most  significant  elements  of  an 
observation.  This  will  speed  up  the  calculation  and  has  no  influence  on  the  result  as  the  method  always 
finds  every  solution.  But  it  is  a  scenario  dependent  and  perhaps  subjective  decision,  to  define  a  threshold 
for  significance.  Heuristic  or  doctrinal  knowledge  about  deployment  rules,  e.g.  that  certain  units  typically 
operate  jointly  but  they  will  not  intermix  their  respective  areas  of  activity,  can  be  used  as  domain  specific 
constraint.  Such  rules  will  reduce  the  number  of  hypotheses  about  the  classification  of  one  observation 
and  by  this  the  complexity  of  following  processing  steps,  and  the  quality  of  the  result  will  be  enhanced. 
Figure  4  graphically  shows  the  schema  of  the  classification  by  an  example. 
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Figure  4:  Schema  of  Classification. 


3.3  Template-Based  Aggregation 

The  further  analysis  of  the  information  of  the  classified  situation  elements  and  their  fusion  to  higher  level 
situation  objects  is  done  in  three  steps.  Starting  with  the  generation  of  so-called  aggregation-objects,  the 
generation  of  aggregation-hypotheses  follows  and  finally  fused  solutions  are  determined. 

The  premise  for  human  and  automated  aggregation  is  that  there  is  sufficient  knowledge  about  high  level 
situation  objects  or  activities,  so  that  suitable  templates  can  be  developed,  describing  the  structure  and 
standard  behaviour  of  the  engaged  factions  in  the  context  of  the  current  scenario. 

The  main  ideas  of  this  approach  will  be  presented  in  the  following  part  of  the  paper.  A  detailed  description 
of  these  processing  steps  is  given  in  [6]. 


3.3.1  Generation  of  Aggregation-Objects 

Depending  on  the  category  of  information,  to  which  a  lower  level  situation  element  is  associated,  it  is 
compared  to  the  templates  of  higher  level  objects  to  test  whether  it  fits  into  any  part  of  these  general 
descriptions.  The  structure  and  the  attributes  of  the  low  level  object  have  to  match  at  least  parts  of  those  of 
the  higher  level  object.  This  information  processing  is  not  related  to  other  low  level  objects  of  the 
perceived  situation.  It  is  an  analysis  of  the  individual  low  level  element  in  the  context  of  knowledge  given 
by  the  basic  intelligence.  If  this  test  is  passed  the  hypothetical  relationship  is  kept  as  a  new  attribute  of  the 
specific  low  level  object  which  is  then  called  an  ‘aggregation-object’.  Attributes  of  the  low  level  object 
may  be  related  to  different  templates  of  the  high  level  object  and/or,  being  less  significant,  they  may  fit  to 
different  parts  of  a  single  template.  So  the  result  of  the  comparison  may  not  be  unique.  All  resulting 
hypotheses  will  be  kept  and  taken  as  input  for  the  following  aggregation  step. 

For  example,  information  about  moving  vehicles  will  be  compared  with  the  functional  template  of  a  march 
column  shown  in  Table  1.  If  the  type  of  the  vehicles  is  given  then  they  can  be  classified  in  respect  to  the 
type  of  troops  they  may  belong  to  (which  might  be  ambiguous).  This  will  then  allow  to  compare  to  the 
functional  march  column  template  to  deduce  to  which  part  of  a  march  columns  the  vehicle  group  may 
belong  to.  Depending  on  the  type  of  troops  this  may  be  ambiguous  so  that  different  hypotheses  have  to  be 
kept.  Additionally  a  known  spatial  extension  of  the  observed  moving  vehicle  group  or  the  minimal 
extension  of  the  induced  type  of  troops  may  constrain  the  matching  test  with  the  functional  template. 
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3.3.2  Aggregating  Hypotheses 

Aggregation-hypotheses  which  are  generated  from  aggregation-objects  represent  higher  level  abstraction 
objects  of  the  perceived  situation  which  may  be  aggregated  on  their  part  in  a  multiple  level  aggregation 
procedure.  A  new  aggregation-object  either  can  be  integrated  into  an  existing  aggregation-hypothesis,  thus 
updating  it  and  confirming  its  evidence  from  a  military  point  of  view,  or  if  it  does  not  fit  to  any  of  the 
existing  hypotheses,  a  new  one  is  generated. 

Figure  5  shows  as  an  example  the  graph  of  all  aggregation-hypotheses  build  out  of  10  aggregation-objects 
aoi ,  i  =  1(1)10.  The  vertices  represent  the  single  aggregation-objects  aoi?  an  edge  between  two  vertices 
indicates  that  the  two  aggregation-objects  together  match  the  underlying  template.  In  Figure  5  [aoi  -  ao7] 
and  [ao7  -  ao8]  both  are  correct  level- 1  aggregation-hypotheses  enclosing  two  aggregation-objects. 


aoi  ao2  ao3  ao4  &05 


ao6  ao7  ao8  ao9  aoio 


Figure  5:  Graph  of  aggregation-hypotheses. 

Several  aggregation-objects  can  be  aggregated  into  one  hypotheses  only  if  they  all  simultaneously  fit 
the  underlying  template  which  has  to  be  tested  by  a  specific  template  dependent  function.  As  in  Figure  5 
[aoi  -  ao8]  also  is  a  correct  level- 1  aggregation-hypothesis  the  triple  [aoi  -  ao7  -  ao8]  is  a  correct  level-2 
hypothesis.  The  matching  relation  of  aggregation-objects  and  their  respective  template  is  not  necessarily 
transitive,  e.  g.  in  the  case  of  march  columns  on  a  road  net.  Therefore  the  integration  of  a  new  aggregation- 
object  into  an  existing  aggregation-hypothesis  may  require  to  test  the  power  set  of  all  combinations  of  the 
component  objects  of  the  old  aggregation-hypothesis  and  the  new  aggregation-object.  As  in  the  worst  case 
the  number  of  hypotheses  doubles  for  the  integration  of  each  new  aggregation-object  special  effort  is  taken 
on  the  internal  representation  of  the  set  of  hypotheses  and  the  updating  algorithm.  An  efficient  recursive 
algorithm  to  create  new  hypotheses  and  to  update  the  set  of  all  valid  hypotheses  is  given  in  [6]. 

3.3.3  The  Fusion  Step 

The  set  of  all  formal  correct  aggregation-hypotheses  will  contain  contradictory  elements  if  some 
aggregation-objects  have  been  aggregated  concurrently  into  different  hypotheses  which  cannot  be  true  all 
at  the  same  time.  A  military  situation  determination  has  to  give  an  unambiguous  picture  of  the  battlespace, 
which  has  to  be  based  on  a  consistent  set  of  hypotheses  concerning  the  different  situation  aspects 
respectively.  The  main  objective  of  the  fusion  step  is  to  find  all  consistent  sets  of  aggregation-hypotheses. 

Definition'.  Fused  Solution,  Minimal  Fused  Solution 

For  a  given  finite  set  AO  =  {  aoi  |  i  e  1}  of  aggregation-objects  and  the  corresponding  set  AH(AO) 
of  all  formal  correct  aggregation-hypotheses  a  Fused  Solution  FS  is  a  subset  of  AH  which  is  a 
partition  of  AO.  FS  is  called  Minimal,  if  there  exists  no  different  subset  FS*  c=  AH  which  is  a  Fused 
Solution  and  ((FS*  c=  FS)  a  (FS*  ^  FS))  is  true. 
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According  to  this  definition  a  Fused  Solution  FS  has  the  following  properties: 

1)  Each  aggregation-object  is  contained  in  one  of  the  aggregation-hypotheses  in  FS 

2)  All  aggregation-hypotheses  in  FS  mutually  do  not  share  any  element  (aggregation-object) 

Which  aggregation-hypotheses  finally  will  belong  to  the  Fused  Solution  may  depend  on  an  assessment 
which  is  based  on  additional  constraints  such  as  operational  effectiveness.  In  the  case  of  a  march  column 
this  does  mean,  that  it  is  reasonable  to  move  along  the  shortest  path  and  not  to  split  up  into  many  different 
march  columns.  Heuristics  of  this  kind  are  used  to  constrain  the  calculation  of  a  fused  solution.  In  many 
cases  a  minimum  constraint  corresponds  to  military  heuristics  about  effectiveness  and  efficiency  of  the 
deployment  of  forces  which  hold  under  the  general  assumption  of  reasonable  operating  adversary  factions. 
Even  if  no  obvious  inherent  minimum  constraint  is  given,  the  calculation  of  a  minimum  number  of 
hypotheses  which  explains  the  information  situation  is  a  valuable  hint  to  the  minimum  number  of  high 
level  situation  elements  which  are  at  least  necessary  to  give  rise  to  the  perceived  situation  picture.  By  this, 
a  lower  limit  of  the  number  of  high  level  situation  elements  is  given. 

The  determination  of  a  Minimal  Fusion  Solution  can  be  formulated  as  an  abstract  graph  problem  which  is 
not  dependent  from  the  application  domain  and  the  originating  intelligence  process  [8].  Starting  from  the 
graph  representation  of  all  formal  correct  aggregation-hypotheses,  as  it  is  given  e.g.  in  Figure  5, 
the  algorithm  has  to  find  all  minimal  clique  partitions  of  the  graph.  The  requirement  for  cliques 
corresponds  to  the  requirement  that  in  a  correct  aggregation-hypothesis  all  aggregation-objects  mutually 
match  the  underlying  template,  which  is  represented  by  complete  sub-graphs.  For  the  example  graph  in 
Figure  5  the  unique  minimal  solution  is  given  by  Figure  6. 


Figure  6:  Graph  of  a  Minimal  Fusion  Solution. 


The  three  cliques  Ci  =  {  aoi  ,  ao2  ,  ao6 ,  ao7  },  C2  =  {  ao3  ,  ao4  ,  ao8  },  and  C3  =  {  ao5  ,  ao9 ,  aoi0  }  define  a 
partition  of  the  set  AH  of  all  formal  correct  aggregation-hypotheses.  In  general  the  minimal  clique 
partition  problem  has  no  unique  solution.  As  a  result  of  the  fusion  step  we  will  get  the  set  of  all  Minimal 
Fused  Solutions.  In  [8]  it  is  proven  that  the  developed  algorithm,  which  is  based  on  a  Branch-and-Bound 
method,  is  total  and  correct,  which  means  that  every  solution  of  the  minimal  clique  partition  problem  is 
determined  and  every  determined  solution  is  a  Minimal  Fused  Solution. 

The  whole  three-step  process  of  template-based  aggregation  and  fusion  is  intended  to  run  continuously 
and  to  develop  iteratively  for  each  new  aggregation-object  a  new  set  of  aggregation-hypotheses  and  a  new 
set  of  Minimal  Fused  Solutions. 

3.4  Hypotheses  Management 

The  set  of  all  aggregation-hypotheses  which  is  calculated  in  every  iteration  step  contains  every  formal 
correct  solution  of  the  matching  problem.  In  an  operational  fusion  system  the  question  of  maintaining  the 
set  of  all  hypotheses  may  become  important,  because  system  performance  may  decrease  rapidly  if  the 
number  of  hypotheses  increases  by  combinatorial  effects.  Pruning  of  hypotheses  could  be  done  either 
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interactively  under  the  complete  control  of  the  user,  or  automated,  or  as  a  combination  of  both  of  them. 
Some  strategies  for  the  pruning  of  hypotheses  are  discussed  in  [6].  The  central  idea  of  the  pruning  policy 
is  to  prefer  those  hypotheses  which  are  part  of  the  set  of  Minimal  Fused  Solutions  for  several  iteration 
steps.  In  a  military  interpretation  these  hypotheses  are  confirmed  several  times  and  thus  have  gained 
stronger  evidence.  So,  basically  this  is  again  a  domain  specific  heuristic  which  constrains  the  solution 
space. 

4.0  CONCLUSION 

Just  as  in  the  case  of  human  intelligence  processing  also  automated  knowledge-based  information  fusion 
depends  on  the  availability  of  a  minimum  of  basic  intelligence  in  terms  of  knowledge  about  structure, 
equipment  and  default  behaviour  of  the  adversary  factions.  Comparing  classical  high  intensity  conflicts, 
civil  war  conflicts,  crisis  reaction  operations,  terrorist  activities  or  organised  criminal  activities  there  are 
great  differences  in  respect  to  the  type  and  amount  of  information  and  intelligence  which  is  available. 
Especially  at  the  beginning  of  a  new  conflict  even  the  information  aspects  and  according  the  kind  of 
templates  which  are  of  relevance  for  a  situation  determination  and  assessment  may  hardly  be  known. 
Furthermore  the  production  of  intelligence  used  to  be  and  will  be  based  on  a  varying  but  limited  number 
of  categories  of  information  of  basic  intelligence  which  are  supposed  to  be  sufficient  to  understand  the 
situation.  As  a  consequence  a  future  supporting  automated  fusion  system  has  to  be  an  interactive  system 
with  the  human  operator  in  the  loop.  An  interactive  interface  to  a  fusion  system  should  allow  to  introduce 
new  or  to  modify  existing  aggregation  templates  to  tune  the  system  to  the  requirements  of  the  specific 
scenario.  Inappropriate  templates  or  constraints  which  are  caused  by  weak  initial  basic  knowledge  have  to 
be  adapted  to  the  real  situation,  which  was  proven  to  be  feasible  in  our  ongoing  research  project  [3]. 
The  intelligence  officer  as  the  system  user  will  be  responsible  for  this  tuning  and  he  is  responsible  for  the 
selection  of  a  specific  hypothesis  from  a  set  of  formal  correct  Minimal  Fusion  Solutions  which  are  offered 
to  him  by  the  system.  The  presented  knowledge-based  method  for  an  automated  fusion  system  is  intended 
to  give  support  for  human  decisions  not  to  substitute  the  intelligence  officer. 

Intelligence  processing  of  information  which  is  related  to  standard  and  default  behaviour  of  the  observed 
factions  can  be  supported  by  constraint  template-based  matching  methods.  This  approach  is  basically  not 
dependent  from  the  special  scenario.  Procedures  based  on  this  approach  will  determine  all  formal  correct 
results  without  any  subjective  prejudice  and  probably  in  a  time  which  is  much  shorter  as  for  a  human 
analyst.  This  will  free  time  for  the  intelligence  officer  to  do  his  proper  task  which  are  the  more 
sophisticated  processing  steps  of  situation  interpretation  and  impact  assessment.  The  interactive  adaptation 
of  procedures  based  on  the  template-based  method  is  one  of  the  challenging  problems  to  develop  flexible 
information  fusion  systems. 
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Overview 


•  The  role  and  the  processing  of  military  intelligence 

•  A  knowledge  based  approach  to  information  fusion  in 
intelligence  processing 

•  An  example  of  template  based  aggregation  of  information 

•  Requirements  and  conclusion 
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The  Role  of  Military  Intelligence 


Intelligence  provides  the  commander 

•  with  a  judgement  of  his  adversaries’  most  likely  tactics 

•  an  assessment  of  their  capabilities 

•  a  prediction  of  their  intentions 

•  an  assessment  of  the  effects  of  the  environment  on  both 
friendly  and  adversary  operations 

Intelligence 

•  is  the  result  of  a  process  of  subjective  judgement 

•  it  is  not  unequivocal 

•  is  open  to  challenge 
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Steps  of  the  Processing  Phase 


information 


I 


JDL  Data  Fusion  Level  2:  Situation  Awareness 

»  The  type  and  state  of  each  single  situation  element  is 
analysed  and  its  relation  to  other  elements 

»  They  are  aggregated  and  fused  to  higher  level  objects 

»  The  structure  of  the  aggregated  or  fused  objects  as  well  as 
activities  of  the  adversary  forces  are  tried  to  be  recognised 
Keview  or  information  to 


Analysis 


Integration 


significant  facts  for  subsequent 


Selection  of  analysed  information 
and/or  intelligence 

combination  into  a  pattern 

in  the  production  of  further  intel  igence 


▼ 


intelligence 


Interpretation 


Judgement  of  the  significance  of  information 

and/or  intelligence 

in  relation  to  the  current  body  of  knowledge 
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Intelligence  Constraints 


•  Only  general  and  incomplete  basic  information  about  the 
structure,  the  activities  and  the  intent  of  adversaries  may 
be  available 

•  Current  information  based  on  reconnaissance  is  imperfect 

•  The  human  procedures  for  situation  awareness  are 

-  mainly  of  heuristic  nature 

-  based  on  subjective  assessment 

HOW  DO  THEY  SUCCEED? 
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Characteristics  of  Military  Operations 


Military  operations 

•  constitute  a  structured  system 
of  interdependent  activities 

•  are  correlated  to  the  situation 
history  and  development 

•  are  goal  driven 

=>  It  is  possible  to  isolate  and 
identify  specific  operational 
tasks  given  mandatory 
indications 


Oestmann,  Rainer:  Dazu  befehle  ich!  Handbuch  fur  militarische  Fuhrer;  3. 
Auflage  2001,  Walhalla  Verlag,  Regensburg/Berlin 
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Premises  of  Intelligence  Processing 


•  Military  operating  forces  act  by  default  according  to  their 
doctrines,  rules,  and  principles 

thus  revealing  a  characteristic  behaviour 

•  Generalised  representations  of  such  patterns  of  behaviour  or  of 
typical  structure  or  state  are  used  as  templates 

•  Templates  have  characteristic  features 

they  can  be  recognised  by  significant  information 

•  The  real  situation  picture  can  be  deduced  from  recognised 
templates 
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t  * 

Template  Based  Processing 


Planning  Battlespace  Activities 


Templates 
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Example 


Template-Based  Information  Processing  for 

-  Situation  Awareness 


-  (Threat  Assessment;  see  presentation  no  9: 

F.  P.  Lorenz,  J.  Biermann,  A  Man-in-the-Loop  Support  Concept  for  Military  Ambush  Threat 
Analysis  Based  on  Reconnaissance  Reports 
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Processing  Reports  about  Marching  Troops 


Problem:  A  great  number  of  march  reports  unaligned  in  space  and  time 
Question:  How  many  and  which  type  of  march  columns  have  been  observed? 
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Attribute  Template  of  March  Columns 


Deducing  the  matching  part  of  a  march  column  from  observed  equipment 


March  column  :: 

=  Column  head  -- 

-  {march  serial}1- n  — 

--  column  end 

Unit  type 

reconnaissance 
security  forces 

combat  forces 
combat  support  forces 
air  defence,  security 

LOG 

air  defence,  security 

Branch 

armoured 

reconnaissance 

antitank  defence 

ARMR,  RIF,  ENGR 
TKINF, 

MRL  MOTRIF,  ARTY, 
AA  SAM 

transportation 
medical  support 
maintenance 

AA  SAM 

Command  level 

<  company 

<  battalion 

<  company 

Type  of  vehicle 

ARV  APC 

MBT  APC  AAA 

VEH  AAA 
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temporal  axis 


Space-Time  Alignment  of  March  Reports 


At 


t, 


shortest  path 


I  0  V  *  At 


report2 


r 

I- 


I 


r 

l_ 


I 


Component  of  the  spatial  structure  of  a  march  column 
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Aggregation  of  March  Columns 


Classified  report 


Attribute  match 

Aggregation 

object 


Space  /  time 
correlation 
Aggregation  into 
march  template 


March 

hypotheses 


head 
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Fusion:  Determination  of  a  Consistent  Minimal  Solution 


•  Input 

Set  of  single  reports  about 
adversary  elements  and 
/  or  activities 


•  Aggregation 

Template  based  establishment 
of  relations  (inconsistent  set  of 
hypotheses) 


•  Fusion 

Partitioning  the  relation  graph 
into  a  minimal  set  of  cliques 
(consistent  set  of  hypotheses) 
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Result  of  the  Aggregation  of  March  Columns 


Input:  36  reports  on  marching  troops  covering  4  hours  real  time 


Result:  8  march  columns  (6  tank,  1  pioneer,  1  artillery) 
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Requirements  for  Template-Based  Information  Fusion 

The  concept  of  Template  Based  Information  Fusion  to  support  intelligence 
processing  depends  on  the  pre-conditions: 

•  default  reasoning  on  a  closed  world  model  of  the  battlespace 

•  the  ability  to  automatically  understand  relevant  information 


Requirements  for  a  successful  Template  Based  Information  Fusion  are: 

•  machine  readable  representation  of  templates  of  standard  behaviour  and  abilities 
of  foreign  factions 

-  preventive  and  continuous  reconnaissance  to  build  up  knowledge  bases  about 
possible  adversaries 

-  expert  guided  development  of  templates  of  the  standards  and  the  default 
behaviour  of  adversaries 

•  semantic  access  to  current  information  and  basic  intelligence 

-  progress  in  information  extraction  and  text  understanding 

•  interactive  intelligence  processing 

-to  adept  templates  to  the  actual  mission 
-to  select  among  system  developed  hypotheses 
-to  alter  and  update  basic  intelligence 
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Intelligence  Officer 


Data  Browser  Situation  Display 


Basic  Intelligence 

Organisation  structure 
Weapon  &  Equipment 
Geography 
Route  system 


Hypotheses 

Browser 


Perceived  Situation 


Perceived  opposing  forces 
Hypotheses  on  tactical  situation  elements 
Hypotheses  on  threats 


Template 

Editors 


Expert  Knowledge 


Templates 

-  Activities 

-  Spatial  deployment 
Doctrines 

Battlespace  Ontology 


Classification  /  Identification 
Template  Based  Processing 


C4ISR  System 
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Conclusion 


We  hold  knowledge-based  information  fusion  to  be  a 
sustainable  approach  to  give  support  to 
context  sensitive  intelligence  processing! 
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Discussion 


QUESTIONS? 
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